<html>
<head>
<meta http-equiv="Content-Type" content="text/html; charset=UTF-8">
<title>Spirit.Lex Tutorials Overview</title>
<link rel="stylesheet" href="../../../../../../../doc/src/boostbook.css" type="text/css">
<meta name="generator" content="DocBook XSL Stylesheets V1.79.1">
<link rel="home" href="../../../index.html" title="Spirit 2.5.8">
<link rel="up" href="../tutorials.html" title="Spirit.Lex Tutorials">
<link rel="prev" href="../tutorials.html" title="Spirit.Lex Tutorials">
<link rel="next" href="lexer_quickstart1.html" title="Quickstart 1 - A word counter using Spirit.Lex">
</head>
<body bgcolor="white" text="black" link="#0000FF" vlink="#840084" alink="#0000FF">
<table cellpadding="2" width="100%"><tr>
<td valign="top"><img alt="Boost C++ Libraries" width="277" height="86" src="../../../../../../../boost.png"></td>
<td align="center"><a href="../../../../../../../index.html">Home</a></td>
<td align="center"><a href="../../../../../../../libs/libraries.htm">Libraries</a></td>
<td align="center"><a href="http://www.boost.org/users/people.html">People</a></td>
<td align="center"><a href="http://www.boost.org/users/faq.html">FAQ</a></td>
<td align="center"><a href="../../../../../../../more/index.htm">More</a></td>
</tr></table>
<hr>
<div class="spirit-nav">
<a accesskey="p" href="../tutorials.html"><img src="../../../../../../../doc/src/images/prev.png" alt="Prev"></a><a accesskey="u" href="../tutorials.html"><img src="../../../../../../../doc/src/images/up.png" alt="Up"></a><a accesskey="h" href="../../../index.html"><img src="../../../../../../../doc/src/images/home.png" alt="Home"></a><a accesskey="n" href="lexer_quickstart1.html"><img src="../../../../../../../doc/src/images/next.png" alt="Next"></a>
</div>
<div class="section">
<div class="titlepage"><div><div><h4 class="title">
<a name="spirit.lex.tutorials.lexer_tutorials"></a><a class="link" href="lexer_tutorials.html" title="Spirit.Lex Tutorials Overview"><span class="emphasis"><em>Spirit.Lex</em></span>
        Tutorials Overview</a>
</h4></div></div></div>
<p>
          The <span class="emphasis"><em>Spirit.Lex</em></span> library implements several components
          on top of possibly different lexer generator libraries. It exposes a pair
          of iterators, which, when dereferenced, return a stream of tokens generated
          from the underlying character stream. The generated tokens are based on
          the token definitions supplied by the user.
        </p>
<p>
          Currently, <span class="emphasis"><em>Spirit.Lex</em></span> is built on top of Ben Hanson's
          excellent <a href="http://www.benhanson.net/lexertl.html" target="_top">Lexertl</a>
          library (which is a proposed Boost library). <a href="http://www.benhanson.net/lexertl.html" target="_top">Lexertl</a>
          provides the necessary functionality to build state machines based on a
          set of supplied regular expressions. But <span class="emphasis"><em>Spirit.Lex</em></span>
          is not restricted to be used with <a href="http://www.benhanson.net/lexertl.html" target="_top">Lexertl</a>.
          We expect it to be usable in conjunction with any other lexical scanner
          generator library, all what needs to be implemented is a set of wrapper
          objects exposing a well defined interface as described in this documentation.
        </p>
<div class="note"><table border="0" summary="Note">
<tr>
<td rowspan="2" align="center" valign="top" width="25"><img alt="[Note]" src="../../../images/note.png"></td>
<th align="left">Note</th>
</tr>
<tr><td align="left" valign="top"><p>
            For the sake of clarity all examples in this documentation assume <span class="emphasis"><em>Spirit.Lex</em></span>
            to be used on top of <a href="http://www.benhanson.net/lexertl.html" target="_top">Lexertl</a>.
          </p></td></tr>
</table></div>
<p>
          Building a lexer using <span class="emphasis"><em>Spirit.Lex</em></span> is highly configurable,
          where most of this configuration is done at compile time. Almost all of
          the configurable parameters have generally useful default values, allowing
          project startup to be a easy and straightforward task. Here is a (non-complete)
          list of features you can tweak to adjust the generated lexer instance to
          the actual needs:
        </p>
<div class="itemizedlist"><ul class="itemizedlist" style="list-style-type: disc; ">
<li class="listitem">
              Select and customize the token type to be generated by the lexer instance.
            </li>
<li class="listitem">
              Select and customize the token value types the generated token instances
              will be able to hold.
            </li>
<li class="listitem">
              Select the iterator type of the underlying input stream, which will
              be used as the source for the character stream to tokenize.
            </li>
<li class="listitem">
              Customize the iterator type returned by the lexer to enable debug support,
              special handling of certain input sequences, etc.
            </li>
<li class="listitem">
              Select the <span class="emphasis"><em>dynamic</em></span> or the <span class="emphasis"><em>static</em></span>
              runtime model for the lexical analyzer.
            </li>
</ul></div>
<p>
          Special care has been taken during the development of the library that
          optimal code will be generated regardless of the configuration options
          selected.
        </p>
<p>
          The series of tutorial examples of this section will guide you through
          some common use cases helping to understand the big picture. The first
          two quick start examples (<a class="link" href="lexer_quickstart1.html" title="Quickstart 1 - A word counter using Spirit.Lex">Lex
          Quickstart 1 - A word counter using <span class="emphasis"><em>Spirit.Lex</em></span></a>
          and <a class="link" href="lexer_quickstart2.html" title="Quickstart 2 - A better word counter using Spirit.Lex">Lex Quickstart
          2 - A better word counter using <span class="emphasis"><em>Spirit.Lex</em></span></a>)
          introduce the <span class="emphasis"><em>Spirit.Lex</em></span> library while building two
          stand alone applications, not being connected to or depending on any other
          part of <a href="http://boost-spirit.com" target="_top">Spirit</a>. The section
          <a class="link" href="lexer_quickstart3.html" title="Quickstart 3 - Counting Words Using a Parser">Lex Quickstart 3
          - Counting Words Using a Parser</a> demonstrates how to use a lexer
          in conjunction with a parser (where obviously the parser is built using
          <span class="emphasis"><em>Spirit.Qi</em></span>).
        </p>
</div>
<table xmlns:rev="http://www.cs.rpi.edu/~gregod/boost/tools/doc/revision" width="100%"><tr>
<td align="left"></td>
<td align="right"><div class="copyright-footer">Copyright © 2001-2011 Joel de Guzman, Hartmut Kaiser<p>
        Distributed under the Boost Software License, Version 1.0. (See accompanying
        file LICENSE_1_0.txt or copy at <a href="http://www.boost.org/LICENSE_1_0.txt" target="_top">http://www.boost.org/LICENSE_1_0.txt</a>)
      </p>
</div></td>
</tr></table>
<hr>
<div class="spirit-nav">
<a accesskey="p" href="../tutorials.html"><img src="../../../../../../../doc/src/images/prev.png" alt="Prev"></a><a accesskey="u" href="../tutorials.html"><img src="../../../../../../../doc/src/images/up.png" alt="Up"></a><a accesskey="h" href="../../../index.html"><img src="../../../../../../../doc/src/images/home.png" alt="Home"></a><a accesskey="n" href="lexer_quickstart1.html"><img src="../../../../../../../doc/src/images/next.png" alt="Next"></a>
</div>
</body>
</html>
